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1.  Introduction 


1.1  Background 

Typically,  when  interface  designers  talk  about  scalable  interfaces,  they  are  referring  to  a  design 
that  ensures  development  considers  the  requirement  to  change  over  time.  For  this  experiment, 
we  are  concentrating  on  a  more  narrow  definition  of  scalability.  We  are  concerned  with  the 
ability  to  scale  down  interfaces  in  terms  of  size  and  weight  and  the  effect  of  this  scaling  on  the 
cognitive  load  of  robotic  operators.  Soldiers  operate  in  a  large  range  of  environments,  from  the 
relatively  stable  and  spacious  environment  of  a  tactical  operations  center  (TOC),  to  the  cramped 
and  constantly  moving  environment  of  a  vehicle,  to  the  rugged  and  challenging  environment  of 
the  dismounted  Soldier.  All  of  these  environments  impose  different  demands  on  the  size  and 
configuration  of  the  robotic  interface.  For  example,  a  dismounted  Soldier  cannot  carry  the 
relatively  large  controller  that  would  be  appropriate  for  use  in  a  TOC.  This  type  of  interface 
scalability  is  very  important  because  it  ensures  that  training  transfer  is  easily  achieved  across 
environments  and  that  interfaces  can  be  tailored  to  specific  environments  and  conditions.  The 
operational  definition  of  scalability  used  in  this  experiment  is  as  follows:  “The  transmission  of 
critical  information  to  the  Soldier  tailored  for  each  level  of  combat  to  ensure  mission  success 
while  maximizing  survivability  by  minimizing  equipment  requirements;  minimizing  multitasking 
workload,  maximizing  situation  understanding,  and  maximizing  aerial  and  ground  robotic 
mission  effectiveness”  (Merlo,  2006).  A  similar  definition  is  “The  tailored  reception  and 
transmission  of  mission  essential  infonnation  at  the  appropriate  level  for  the  Soldier,  to  ensure 
mission  success  while  maximizing  the  survivability  and  lethality  through  the  synergistic 
interaction  of  equipment  requirements,  appropriate  cognitive  workload,  situation  awareness  and 
understanding  for  oneself  and  others  connectivity  of  distributed  intelligent  agents”  (Barnes, 
2006). 

The  key  to  ensuring  that  a  system  is  scalable  is  to  consider  not  only  the  range  of  devices  that 
Soldiers  will  use,  but  also  their  context  of  use.  A  familiar  example  of  a  context-sensitive 
application  is  e-mail  access.  Typically,  users  have  used  their  desktop  computers  to  access  email 
but  more  and  more  frequently,  they  are  now  using  personal  digital  assistants  (PDAs)  and  cell 
phones  to  do  the  same  job  when  they  are  outside  of  their  offices.  Robots  can  be  teleoperated 
through  a  wide  variety  of  control  media,  ranging  from  hand-held  devices  such  as  PDA  systems 
(Fong  et  ah,  2004;  Quigley  et  al.,  2004)  and  cellular  phones  (Sekmen  et  al.,  2003)  to  multiple 
panel  displays  with  control  devices  such  as  joysticks,  wheels,  and  pedals  (Kamsickas,  2003). 

The  input  devices  for  smaller  interfaces  are  quite  different  from  those  found  in  offices,  vehicles, 
or  other  environments  in  which  the  operator  is  not  responsible  for  carrying  the  device  and  can 
potentially  impact  the  complexity  of  operation,  the  speed  of  operation,  and  the  accuracy  of  input. 
Designing  for  the  “optimum”  input  device  size  makes  sense  as  long  as  the  interface  environment 
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and  operating  conditions  can  be  specified  in  advance.  However,  if  an  input  device  is  designed  so 
that  it  is  only  practical  in  one  environment,  it  may  be  completely  unusable  in  another  and  for 
future  tasks  or  unexpected  conditions.  Other  factors  of  scalability  that  will  be  addressed  in 
subsequent  experiments  include  structure  or  organization  of  the  content  of  an  interface  (a 
desktop  computer  may  use  a  presentation  that  is  optimized  when  using  a  high-resolution  monitor, 
while  a  user  of  a  PDA  might  view  the  information  in  a  text-only  presentation)  and  information 
requirements  of  users  in  different  environments. 

Space  to  incorporate  controls  on  small-size  controllers  is  very  limited.  Miniaturizing  individual 
input  controls  as  controller  sizes  get  smaller  is  not  always  an  option,  as  Soldiers  have  to  be  able 
to  operate  them  individually  without  accidental  activation  of  adjacent  controls  while  wearing 
gloves.  Thus,  the  designer  of  controllers  must  be  creative  during  function  mapping;  this  often 
drives  them  to  using  multifunction  controls.  The  problems  with  multifunction  controls  are  that 
they  can  increase  control  activation  time  and  increase  the  cognitive  complexity  of  the  controller. 
Other  creative  approaches  to  controller  size  reduction  that  have  been  developed  in  the  past  are 
sketch  interfaces  (Skubic  et  al.,  2003;  Setalaphruk  et  al.,  2005),  voice  recognition  and  synthesis 
systems  (Chen  et  al.,  2006),  and  hands-free  systems  (Veronka  and  Nestor,  2001).  However, 
these  novel  controls  often  present  problems  of  their  own  and  are  still  being  refined.  In  the 
meantime,  there  is  an  immediate  requirement  for  controls  for  dismounted  Soldiers  to  teleoperate 
robots.  There  is  also  not  a  great  deal  of  empirical  data  on  the  impact  of  decreasing  the  size  of 
controllers  for  dismounted  operations. 

This  experiment  was  the  second  in  a  series  of  experiments  designed  to  investigate  current  and 
future  options  for  scaling  robotic  controllers  specifically  for  use  by  dismounted  Soldiers.  In  the 
first  experiment,  controller  type,  workload,  and  usability  were  evaluated  (Pettitt  et  al.,  2008). 

The  controllers  used  were  a  multifunction  control  mounted  on  a  weapon,  a  gaming  controller 
with  reduced  control  sizes,  and  the  larger  robot  legacy  controller.  Findings  indicated  that  the 
multifunctional  controller  was  more  difficult  to  learn  how  to  use  than  the  controller  with  the 
reduced  control  sizes  because  switching  between  functions  was  time  consuming  and  confusing. 
Also,  many  simultaneous  functions  could  not  be  accomplished  with  the  multifunctional 
controller.  In  this  experiment,  we  chose  the  gaming  controller  used  in  the  previous  experiment 
as  the  baseline  condition.  We  compared  the  gaming  controller  to  a  touch-screen  controller  that 
allows  the  display  and  control  functions  to  be  combined  in  one  space.  Both  controllers  were 
programmed  to  provide  the  same  functions.  Pretest  experimentation  was  performed  to  ensure 
that  the  functional  mapping  of  each  of  the  controller  was  as  effective  as  possible.  The  tradeoffs 
between  interface  size  and  weight  with  these  controllers  vs.  input  speed,  accuracy,  training  time, 
and  cognitive  load  were  examined. 
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1.2  Objective 

The  goal  of  this  research  was  to  investigate  the  ability  of  Soldiers  to  use  a  touch-screen  robotic 
controller.  Soldiers’  performance  using  an  Android  touch-screen  controller  was  compared  with 
their  performance  using  a  baseline  joystick  controller. 

1.3  Overview  of  Experiment 

This  study  was  a  cooperative  research  effort  between  the  U.S.  Army  Research  Laboratory 
(ARL)/Human  Research  and  Engineering  Directorate  (HRED)  and  ARL’s  Computational  and 
Information  Sciences  Directorate  (CISD).  It  was  an  investigation  of  the  effect  of  controller 
scalability  on  robotic  control  and  took  place  at  Fort  Benning,  GA.  Thirty  Soldiers  from  the 
Officer  Candidate  School  (OCS)  participated  in  the  study.  After  training  on  the  operation  of  the 
PackBot  Robot  system,  each  Soldier  completed  the  indoor  and  outdoor  courses,  once  with  each 
controller  type.  The  sequence  of  controller  type  was  counterbalanced  to  control  for  order  effects. 
Controller  usability  was  evaluated  based  on  objective  perfonnance  data,  data  collector 
observations,  and  Soldier  questionnaires. 


2.  Method 


2.1  Participants 

Thirty  Soldiers  from  the  Ft.  Benning  OCS  participated  in  the  assessment.  The  OCS  participants 
included  Soldiers  with  prior  enlisted  service  with  a  variety  of  backgrounds  and  experience  levels 
as  well  as  those  just  coming  into  the  Army  from  college. 

2.2.1  Pretest  Orientation 

The  Soldiers  were  given  an  orientation  on  the  purpose  of  the  study  and  what  their  participation 
would  involve.  They  were  briefed  on  the  objectives,  procedures,  and  the  robotic  system.  They 
were  also  told  how  the  results  would  be  used  and  the  benefits  the  military  could  expect  from  this 
investigation.  Any  questions  the  subjects  had  regarding  the  study  were  answered. 

2.2  Apparatus  and  Instruments 

2.2.1  PackBot  Robot 

The  PackBot  Explorer  Robot  is  a  variant  of  the  PackBot  which  has  been  fielded  to  Operation 
Iraqi  Freedom  and  Operation  Enduring  Freedom  since  2003.  The  platform  is  a  man-portable 
small  unmanned  ground  vehicle  which  can  be  used  for  reconnaissance  tasks  including  entering 
and  securing  areas  that  are  either  inaccessible  or  too  dangerous  for  humans  (see  figure  1).  The 
PackBot  Explorer  payload  has  a  rotating  pan  and  tilt  head  equipped  with  multiple  cameras, 
which  was  kept  in  a  fixed  position  for  this  experiment.  The  robot  was  also  equipped  with  a 
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Hokuyo  laser  detection  and  ranging  (LADAR)  sensor  used  to  provide  obstacle  detection  and 
avoidance  capabilities  (O’Brien  et  al.,  2010).  These  capabilities  were  available  to  the  operator 
during  the  indoor  trials  of  the  experiment. 


Figure  1.  iRobot  PackBot  explorer  robot. 

2.2.2  Robotic  Vehicle  Controllers 

Two  operator  control  units  (OCUs)  were  used  during  this  experiment.  The  first  was  an  Itronix 
tablet  that  was  carried  in  the  Soldiers’  backpacks  and  connected  to  a  Microsoft  Xbox*  360  game 
controller  (see  figure  2)  and  a  handheld  Android  phone  with  a  touch-screen  interface  (see  figure 
3).  The  second  configuration  was  just  the  Android  phone.  The  Android  video  display  (3.17 
inches  in  diagonal  with  320  x  480  pixel  resolution)  was  used  to  view  the  video  feed  from  the 
Packbot  when  both  types  of  configurations  were  used.  This  was  done  so  that  the  size  of  the 
video  display  was  held  constant  and  would  not  impact  operator  performance  with  either  of  the 
two  controller  types  being  evaluated. 


Figure  2.  Xbox  360  controller. 


Xbox  is  a  trademark  of  Microsoft  Corporation. 
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Figure  3.  Android  phone. 

Under  the  first  configuration,  the  Xbox  360  controller  was  used  as  the  source  of  input  from  the 
user,  and  the  Android  phone  was  used  as  the  video  output  device.  The  anticipated  primary 
advantage  of  this  configuration  was  that  the  Xbox  controller  is  optimized  for  user  input.  Video 
games  have  become  ubiquitous  in  modern  society;  as  such,  Microsoft  and  other  game  companies 
have  advanced  controller  design  over  the  course  of  many  years  of  research  and  investment.  In 
addition,  many  Soldiers  were  expected  to  have  some  experience  with  the  controller  from 
personal  use.  The  anticipated  primary  disadvantages  of  this  configuration  were  the  weight  and 
size.  This  configuration  requires  the  Itronix  tablet  computer  that  the  Soldier  carried  in  a 
backpack.  In  addition,  the  Xbox  controller  requires  two  hands  to  use  and  is  an  additional  piece 
of  equipment  that  must  be  carried. 

The  second  OCU  configuration  used  only  the  Android  phone  for  both  control  and  video.  The 
control  portion  of  the  Android  interface  involves  four  buttons  and  a  virtual  joystick  controlled 
through  the  touch  interface  of  the  phone.  The  virtual  joystick  is  operated  by  touch  input  in  a 
bounded  square  immediately  surrounding  the  center  joystick  dot.  Touching  within  the  upper  half 
of  the  square  sends  a  “forward”  command  to  the  robot.  Similarly,  touching  on  the  left  half  of  the 
square  will  send  a  “turn  left”  command  to  the  robot  and  likewise  for  “turn  right”  and  “back  up.” 
The  user  interface  allows  analog  control  of  the  robot.  The  further  from  the  center  the  touch 
input,  the  faster  the  robot  will  move  in  that  direction.  The  primary  anticipated  advantage  of  this 
controller  is  the  small  packaging.  Using  the  Android  OCU  configuration  allows  one -hand 
control  in  addition  to  the  elimination  of  the  backpack,  tablet  computer,  and  controller. 

Anticipated  disadvantages  of  this  configuration  included  a  higher  demand  in  user  precision.  The 
virtual  joystick  has  a  smaller  degree  of  movement  and  does  not  have  the  physical  feedback  given 
by  the  thumbstick  of  the  Xbox  controller. 

2.2.3  Outdoor  Robotic  Driving  Course 

The  robotic  course  (figure  4)  was  approximately  200  m  long,  1  m  wide,  and  clearly  marked  with 
white  engineer  tape  on  the  left  and  right  sides.  The  Soldiers  teleoperating  the  robot  used  a 
bounding  movement  to  negotiate  the  course  along  with  the  robot.  The  course  was  designed  with 
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obstacles  that  masked  the  Soldiers’  view  of  the  robot,  forcing  them  to  maneuver  the  robot  using 
only  the  driving  camera  and  display.  The  obstacles  included  a  tunnel,  hills,  a  covered  area,  and 
tents.  Three  transition  points  were  marked  with  red  flags.  The  transition  points  marked  the 
locations  where  the  Soldiers  maneuvered  the  robot  from  a  location  behind  them  to  one  in  front  of 
them  in  order  to  reconnoiter  the  lane. 


< 


Figure  4.  Outdoor  robotic  course. 


2.2.4  Indoor  Robotic  Driving  Course 

The  building  reconnaissance  course  (figure  5)  was  established  at  the  HRED  facility  at  the 
McKenna  military  operations  in  urban  terrain  site  at  Fort  Benning.  It  consisted  of  a  one-story 
building  with  one  large  partitioned  room.  Tables,  chairs,  computers,  and  other  furnishings  were 
placed  at  varying  locations  along  the  reconnaissance  route  in  order  to  increase  the  difficulty  of 
negotiating  the  route. 

Soldier  operators  were  located  out  of  the  line  of  sight  of  the  robot  at  a  stationary  position  inside  a 
tent  near  the  building.  Based  on  data  obtained  from  a  LADAR  sensor,  the  robot’s  obstacle 
detection  and  avoidance  algorithm  was  used  during  the  indoor  trials.  This  algorithm  is  designed 
to  assist  in  teleoperation  by  detecting  obstacles  within  the  path  of  the  robot  and  navigating  the 
robot  to  open  areas.  The  intended  result  is  to  provide  an  obstacle  avoidance  behavior  to  assist 
the  operator  in  space-constrained  navigation  (Pierce  et  ah,  2010). 
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Figure  5.  Indoor  robotic  driving  course. 

2.2.5  The  National  Aeronautics  and  Space  Administration-Task  Load  Index 
(NASA-TLX) 

The  NASA-TLX  requires  the  user  to  rate  the  workload  of  a  device  on  a  number  of  different 
scales  and  to  assign  an  importance  weight  to  each  scale.  The  scores  on  the  workload  scales 
(mental,  physical,  temporal,  performance,  effort,  and  frustration)  can  be  combined  in  an  overall 
workload  score  (Hart  and  Staveland,  2008). 

2.2.6  Questionnaires 

The  questionnaires  were  designed  to  elicit  Soldiers’  opinions  about  their  performance  and 
experiences  with  each  of  the  controller  systems.  The  questionnaires  asked  the  Soldiers  to  rate 
the  devices  on  a  7-point  semantic  differential  scale  ranging  from  “extremely  bad/difficult”  to 
“extremely  good/easy.”  Questionnaires  were  administered  to  each  Soldier  at  the  end  of  each 
iteration  (with  each  type  of  controller)  and  at  the  end  of  the  experiment.  Questionnaires  were 
also  used  to  gather  information  concerning  the  Soldiers’  demographic  data,  robotic  experience, 
and  physical  characteristics  that  might  affect  their  ability  to  operate  the  robot. 

2.3  Procedures 

2.3.1  Soldier  Orientation 

The  experiment  Soldiers  reported  in  groups  of  six  for  one  day  each,  from  0800  to  1700  hours 
daily.  Upon  arrival,  they  received  a  roster  number  used  to  identify  them  throughout  the 
evaluation.  The  Soldiers  completed  an  informed  consent  form,  medical  status  form,  and  a 
demographics  questionnaire.  They  were  given  an  oral  operations  order  that  explained  the  robotic 
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mission  that  they  would  undertake  during  the  experiment.  The  training  and  robotic  courses  were 
also  explained,  and  any  questions  the  Soldiers  had  concerning  the  experiment  were  answered. 

2.3.2  Training 

A  representative  from  CISD  trained  the  Soldiers  on  the  use  of  the  PackBot.  Soldiers  practiced 
teleoperating  the  robot  on  the  same  courses  used  during  the  experiment  to  help  mitigate  learning 
effects.  They  were  trained  on  each  controller  just  before  executing  the  course  with  that 
controller.  Soldiers  were  considered  trained  once  they  were  able  to  complete  the  training  course 
without  assistance.  The  average  training  time  required  was  35  min.  Questionnaires  concerning 
the  amount  of  practice  time  given,  the  level  of  detail  presented,  and  the  adequacy  of  training  aids 
were  administered  at  the  completion  of  training. 

2.3.3  Robotic  Course  Iterations 

Soldiers  negotiated  each  of  the  robotic  driving  courses  twice,  once  using  each  of  the  OCU 
configuration  types.  Both  the  robot  and  the  Soldiers  teleoperating  the  robot  negotiated  the 
outdoor  course.  The  Soldier  advanced  through  the  course  using  a  bounding  method,  alternating 
between  foot  movement,  and  teleoperating  the  robot  from  a  stationary  position.  The  Soldier 
moved  to  a  predetennined  transition  point  first  and  then  maneuvered  the  robot  past  his  position 
to  the  next  transition  point.  He  moved  up  to  the  next  transition  point  before  continuing 
teleoperation  of  the  robot.  The  operator  continued  bounding  between  transition  points  until 
reaching  the  end  of  the  course.  Upon  completion  of  the  outdoor  course,  the  operator  moved  to  a 
stationary  control  station  set  up  outside  the  indoor  driving  course  and  executed  the  indoor  driving 
course  using  the  same  controller.  Soldiers  completed  two  iterations,  one  with  each  controller 
type  on  each  course  for  four  runs.  Data  collectors  recorded  the  times  to  complete  each  course, 
the  number  of  times  the  robot  went  off  course  (outside  the  boundaries),  and  the  number  of 
driving  errors  committed.  The  operator  was  given  a  “forward”  driving  error  for  causing  the 
robot  to  hit  an  object  when  maneuvering  the  robot  forward.  If  an  object  was  hit  when  the  robot 
was  reversing,  a  “rear”  driving  error  was  recorded.  Upon  completing  each  iteration,  the  Soldiers 
were  given  a  questionnaire  designed  to  assess  their  perfonnance  and  experiences  with  each  of  the 
control  systems.  The  participants  also  completed  the  NASA-TLX.  Half  of  the  Soldiers  used  the 
Android  controller  first  and  half  used  the  Xbox  360  controller  first. 

2.3.4  End  of  Experiment  Questionnaire 

After  completing  both  courses,  the  Soldiers  completed  an  end-of-experiment  questionnaire  that 
compared  each  of  the  controllers  on  a  number  of  characteristics.  They  also  completed 
questionnaires  concerning  the  information  requirements  for  teleoperating  the  robot. 

2.4  Experimental  Design 

2.4.1  Independent  Variable  (Within  Subjects) 

•  Controller  type 
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2.4.2  Dependent  Variables 

•  NASA-TLX  workload  scores 

•  Course  completion  times 

•  The  number  of  times  the  driver  went  off  course 

•  The  number  of  driving  errors  on  the  course  (forward  and  rear) 

•  Data  collector  comments 

•  Questionnaire  comments 
2.5  Data  Analysis 

All  objective  data  were  analyzed  using  paired  sample  /-tests.  Cohen’s  d,  a  measure  of  effect 
size,  was  computed  for  each  t  value.  Cohen’s  d  is  the  difference  between  the  means  divided  by 
the  pooled  standard  deviation.  Sequence  effects  were  controlled  through  the  counterbalanced 
order  of  the  experimental  design.  Soldier  questionnaire  data  were  analyzed  using  descriptive 
statistics  on  the  subjective  ratings. 


3.  Results 


3.1  Demographics 

The  Soldiers  ranged  in  rank  from  E4  to  E5.  The  average  age  of  the  Soldiers  was  30  years,  and 
the  average  time  in  service  was  33  months.  None  of  the  Soldiers  had  any  prior  military 
experience  in  teleoperating  a  ground  unmanned  robot.  Detailed  responses  to  the  demographics 
questionnaire  are  available  in  appendix  A. 

3.2  Training 

The  participants  rated  the  training  as  being  very  good  for  both  controller  types.  Learning  to 
operate  the  controls  and  drive  the  robot  was  easy  for  both  controllers.  The  Soldiers  indicated 
that  the  hardest  task  to  learn  was  controlling  the  robot’s  speed  when  turning  using  the  Android 
controller.  Several  Soldiers  commented  that  the  simplicity  of  the  Android  made  it  easy  to  learn 
to  use.  Detailed  responses  to  the  training  questionnaire  are  available  in  appendix  B. 

3.3  Robotic  Course  Results 

Table  1  and  figure  6  show  the  mean  course  completion  times  for  the  two  controllers.  Mean  times 
with  the  Xbox  were  significantly  faster  than  the  mean  times  for  the  Android  on  both  the  outdoor 
[/( 29)  =  6.90,/?  <  0.001,  d=  1.53]  and  indoor  [/(29)  =  9.15, p  <  0.001,  d=  1.89]  courses. 
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Table  1.  Mean  course  completion  times  (minis). 


Xbox 

Android 

Outdoor 

Indoor 

Outdoor 

Indoor 

Mean 

4:35 

2:42 

7:20 

4:38 

SD 

0:59 

0:38 

2:21 

1:17 

Note:  SD  =  standard  deviation. 


Figure  6.  Mean  course  completion  times  with  95%  confidence  intervals. 

The  mean  number  of  off-course  errors  for  both  controllers  on  the  outdoor  course  is  shown  in 
table  2  and  in  figure  7.  There  were  significantly  fewer  errors  with  the  Xbox  than  with  the 
Android,  29)  =  5.53, p  <  0.001,  d  =  1.15. 

Table  2.  Mean  number  of  off-course  errors, 
outdoor  course. 


Xbox 

Android 

Mean 

4.23 

10.17 

SD 

3.76 

6.22 
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Figure  7.  Mean  off  course  errors  with  95%  confidence  intervals. 


Table  3  and  figures  8  and  9  show  the  mean  number  of  forward-  and  rear-driving  errors  on  the 
indoor  and  outdoor  courses.  As  shown  in  table  4,  there  were  significantly  fewer  forward  and  rear 
errors  on  both  courses  with  the  Xbox  controller. 

Table  3.  Mean  number  of  driving  errors. 


Xbox 

Android 

Outdoor 

Indoor 

Outdoor 

Indoor 

Forward 

Rear 

Forward 

Rear 

Forward 

Rear 

Forward 

Rear 

Mean 

3.20 

0.20 

1.03 

2.13 

7.37 

1.47 

2.97 

3.90 

SD 

3.43 

0.41 

1.77 

2.80 

4.24 

1.63 

2.40 

2.96 

11 
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Table  4.  Summary  of  t-tests,  driving  errors. 


Variable 

t 

df 

P 

d 

Outdoor  forward  driving  errors 

-5.19 

29 

<0.001a 

1.08 

Outdoor  rear  driving  errors 

-4.13 

29 

o.oor 

1.06 

Indoor  forward  driving  errors 

-5.12 

29 

<0.001a 

0.92 

Indoor  rear  driving  errors 

-3.4 

29 

0.002a 

0.61 

ap  <0.05,  two-tailed. 


It  is  interesting  to  note  that  for  the  indoor  trials,  more  rear  errors  were  made  with  both  types  of 
controllers  than  forward  errors.  This  is  because  the  guarded  teleoperation  worked  mostly  in  the 
front  of  the  robot  (approximately  240°).  When  errors  without  the  guarded  teleoperation 
(outdoors  forward  and  indoors  and  outdoors  rear)  are  compared  to  errors  with  the  guarded 
teleoperation  (indoors  forward),  there  is  an  indication  of  the  benefit  provided  by  the  guarded 
teleoperation  mode. 

3.4  NASA-TLX  Results 

Table  5  shows  the  means  of  the  NASA-TLX  scales  as  well  as  the  total  workload  means.  The 
paired  sample  t-tests  summarized  in  table  6  indicate  that  the  workload  was  significantly  higher 
for  the  Android  relative  to  the  Xbox  on  the  mental,  effort,  and  frustration  scales  and  in  terms  of 
total  workload.  The  physical  workload  was  quite  small  for  both  controllers.  The  total  workload 
means  are  shown  in  figure  10,  and  the  scale  means  are  shown  in  figure  11. 

Table  5.  Scale  means  and  total  workload  means,  NASA-TLX. 


Scale 

Android 

Xbox 

Mean 

SD 

Mean 

SD 

Mental 

67.0 

19.9 

40.2 

22.0 

Physical 

33.7 

26.7 

23.7 

18.6 

Temporal 

62.7 

15.9 

43.0 

22.4 

Performance 

53.3 

21.7 

29.5 

21.8 

Effort 

59.3 

23.7 

30.8 

18.0 

Frustration 

63.0 

25.0 

26.8 

19.6 

Total  workload 

61.4 

17.6 

36.2 

16.4 

Table  6.  Summary  of  t-tests,  NASA-TLX. 


Scale 

t 

df 

P 

d 

Mental 

7.49 

29 

<o.oor 

1.28 

Physical 

2.06 

29 

0.048a 

0.44 

Temporal 

6.26 

29 

<o.oor 

1.01 

Performance 

4.65 

29 

o.oor 

1.09 

Effort 

7.06 

29 

o.oor 

1.36 

Frustration 

9.55 

29 

o.oor 

1.61 

Total 

8.60 

29 

o.oor 

1.48 

ap  <0.05,  two-tailed. 
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Figure  10.  NASA-TLX  total  workload  means  with  95%  confidence  intervals. 


aoo 
70  0 


Figure  11.  NASA-TLX  scale  means  with  95%  confidence  intervals. 
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3.5  Questionnaire  Results 


Upon  completion  of  the  outdoor  and  indoor  courses,  Soldiers  were  asked  to  rate  their  ability  to 
perform  the  robotic  maneuver  tasks  using  each  controller.  The  tasks  were  rated  using  a  7-point 
scale,  with  1  being  extremely  difficult  and  7  being  extremely  easy.  Table  7  shows  the  maneuver 
tasks  and  ratings  for  each  controller.  All  of  the  tasks  were  easier  to  perfonn  with  the  Xbox  than 
with  the  Android  controller. 

Table  7.  Maneuver  task  ratings. 


Maneuver  Tasks 

Mean  Response 

Xbox 

Android 

Move  in  the  correct  direction  (outside  unguarded  configuration) 

5.50 

3.27 

Move  in  the  correct  direction  (inside  guarded  configuration) 

5.57 

3.37 

Avoid  obstacles  (outside  unguarded  configuration) 

5.13 

3.30 

Avoid  obstacles  (inside  guarded  configuration) 

5.20 

3.63 

Identify  any  terrain  features  that  might  have  an  adverse  effect  on  the  ability  of 
the  robot  to  maneuver  through  the  course 

4.90 

4.07 

Anticipate  whether  the  turn  radius  of  the  vehicle  will  allow  a  turn 

4.70 

3.70 

Identify  if  you  are  on  the  course 

4.83 

4.32 

Maintain  control  when  driving  at  slowest  speeds 

6.07 

4.30 

Maintain  control  when  driving  at  medium  speeds 

5.53 

3.55 

Maintain  control  when  driving  at  fastest  speeds 

4.47 

2.33 

Return  to  the  correct  route  after  navigating  around  obstacles 

5.07 

3.67 

Overall  ability  to  perform  driving  tasks 

5.50 

3.47 

Overall  controller  rating 

5.93 

3.72 

Twenty-six  of  the  30  participants  stated  they  preferred  the  Xbox  controller  to  the  Android 
controller.  Two  preferred  the  Android,  and  two  had  no  preference.  Several  Soldiers  commented 
that  their  preference  for  the  Xbox  controller  was  based  on  their  familiarity  from  previously 
playing  video  games.  Others  stated  the  haptic  feedback  they  got  from  manipulating  the  Xbox 
controller  toggle  allowed  them  to  control  the  robot’s  speed  better  than  the  Android’s  touch¬ 
screen  toggle.  Adjusting  to  the  sensitivity  of  the  Android’s  touch-screen  toggle  control  was 
considered  to  be  the  hardest  task  to  leam. 

The  Android  display  was  used  to  view  the  driving  camera  video  in  both  conditions.  For  both 
controller  conditions,  Soldiers  stated  they  had  difficulty  viewing  the  Android’s  screen  in  direct 
sunlight.  The  robot’s  driving  camera  also  provided  a  dark  image  when  cloud  cover  caused  low- 
light  level  conditions.  During  the  Xbox  trials,  Soldiers  were  required  to  either  set  the  display 
down  on  something  or  hold  the  Android  display  with  one  hand  and  manipulate  the  Xbox 
controller  with  the  other.  Many  Soldiers  stated  they  would  prefer  the  display  to  be  attached  to 
the  controller. 

The  Soldiers  liked  the  Android’s  overall  design  and  stated  that  its  simplicity  made  learning  to 
use  it  easy.  They  also  liked  its  compact  size,  light  weight,  and  being  able  to  operate  the  robot 
with  one  hand.  They  did  state  that  the  size  of  the  screen  was  sometimes  a  problem  because  of 
inadvertent  activation  of  adjacent  controls.  One  Soldier  stated  that  it  was  difficult  to  get  the 
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screen  controller  to  register  the  actual  position  of  the  user’s  linger.  The  sensitivity  of  the  touch 
controller  also  made  it  difficult  for  them  to  maintain  smooth,  steady  operation  and  made 
overcorrection  a  common  problem.  Steering  while  maintaining  speed  was  also  identified  as  a 
problem  with  the  touch  controller.  Several  Soldiers  stated  that  their  performance  with  the 
Android  controller  would  most  likely  improve  with  more  practice. 

Detailed  responses  to  the  post-iteration  and  end-of-experiment  questionnaires  are  available  in 
appendices  C  and  D. 


4.  Discussion  and  Recommendations 


The  Xbox  controller  was  demonstrated  to  be  good  for  controlling  robotic  driving  tasks  because 
total  course  completion  times,  driving  errors,  and  off-course  errors  were  significantly  better. 

This  finding  is  consistent  with  the  findings  of  other  military  designers  who  have  successfully 
mapped  the  Xbox  controller  to  robot  driving  functions  (Pettitt  et  ah,  2008;  Cheung,  2008;  Hodge, 
2009;  Wright,  2010).  The  Xbox  controller  has  been  demonstrated  to  be  easy  for  dismounted 
Soldiers  to  use  because  of  its  size,  weight,  and  design  (Pettitt  et  al.,  2008).  The  rubber  thumb 
sticks  are  intuitive  for  driving  the  robot;  this  is  especially  true  for  the  many  Soldiers  who 
frequently  play  video  games  using  the  Xbox  controller. 

The  Android  touch  screen  used  during  this  experiment  was  extremely  sensitive,  creating  many 
driving  errors  and  requiring  the  robot  to  be  backed  up  frequently  for  repositioning.  This 
increased  course  completion  times.  Also,  the  screen  and  buttons  were  fairly  small,  and  many 
errors  were  made  because  of  inadvertent  button  activation.  The  scaled-down  control  also 
contributed  to  the  sensitivity  issue.  The  virtual  joystick  needed  to  control  the  robot  at  both  slow 
speeds  for  maneuverability  and  fast  speeds  to  overcome  terrain  such  as  loose,  wet  grass.  With 
relatively  small  screen  real  estate,  the  extreme  difference  between  minimum  and  maximum 
speed  had  to  be  controlled  within  a  small  area.  Other  more  autonomous  control  approaches  for 
robotic  supervisory  control  might  prove  more  fruitful  for  touch  control.  For  example,  Mark 
Micire  successfully  used  the  Microsoft  Surface  touch  screen  to  guide  swarm  robots  (Saenz, 
2010).  His  touch-screen  program  gave  operators  different  levels  of  control,  many  of  which  were 
supervisory  control  levels.  He  also  used  the  touch  screen  for  more  direct  control  of  individual 
robots.  However,  the  touch  interface  for  direct  human  control  was  much  larger  and  more 
sophisticated  than  the  interface  on  the  Android  used  during  this  experiment.  Haas  et  al.  (2010) 
used  touch  control  for  swam  robots  too.  In  their  experiment,  touch  was  used  to  define  the 
location  of  the  map  object  and  not  for  direct  driving. 

While  the  Android  touch  screen  was  not  as  successful  as  the  Xbox  controller  for  direct  driving, 
other  instantiations  of  touch-screen  control  might  prove  more  successful.  For  example,  a  more 
successful  touch-based  interface  might  require  larger  buttons  to  accommodate  human  fingers  and 
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buttons  that  are  placed  farther  apart  than  the  Android  allowed.  Adams  and  Kaymaz-Keskinpala 
(2004)  and  Keskinpala  et  al.  (2003)  experimented  with  a  PDA-based  touch  interface  for  gloved 
finger  interactions.  This  interface  had  to  have  larger-than-nonnal  touch-screen  buttons  for 
commanding  the  robot;  this  used  a  lot  of  the  PDA  display  space.  The  space  conflict  issue  was 
addressed  by  providing  buttons  that  were  transparent  and,  thus,  they  maximized  the  use  of 
available  space  on  the  display  screen.  Also,  incorporating  haptic  or  audio  feedback  into  the 
touch  screen  could  potentially  assist  the  operator  in  knowing  when  the  control  is  engaged  since 
this  was  a  frequently  cited  problem.  Lundberg  et  al.  (2003)  found  that  the  lack  of  haptic 
feedback  forced  the  user  to  move  his  or  her  vision  from  the  robot  driving  information  to  the 
control  screen.  The  reverse  might  also  be  true  that  if  the  operator  does  not  move  his  or  her  vision 
from  the  driving  display,  it  could  create  more  errors  during  operation.  Thus,  some  type  of 
feedback  should  be  provided. 


5.  Conclusion 


The  Android  could  be  used  to  successfully  teleoperate  a  robot.  The  primary  benefit  is  the  small 
interface  that  minimizes  the  equipment  burden  of  the  Soldier.  Soldiers  provided  positive 
comments  about  its  small  size,  light  weight,  and  one-handed  use,  stating  that  the  fonn  of  the 
controller  was  excellent  for  dismounted  operations.  However,  costs  in  terms  of  time  and  errors 
are  expected  beyond  those  found  with  other  more  physical  type  controls  such  as  the  Xbox.  Other 
larger  touch-screen  interfaces  that  incorporate  haptic  or  auditory  feedback  might  prove  more 
successful  for  direct  teleoperation.  Also,  the  Android  touch  screen  might  be  more  successful 
when  used  for  more  supervisory  control  functions.  Further  testing  with  more  in-depth 
performance  evaluation  could  also  be  used  to  recalibrate  the  virtual  joystick  to  a  more  useable 
status  by  modifying  top  speed,  turning  rate,  and  acceleration. 
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Appendix  A.  Demographics 


This  appendix  appears  in  its  original  form,  without  editorial  change. 


SAMPLE  SIZE  (N)  =  30 


MOS 

RANK 

AGE 

DUTY  POSITION 

09S-  19 

73C-  1 

E-4-8 

30  years 

ocs 

-  12 

18F  -  1 

77F-  1 

E-5-  14 

(mean) 

G4 

-2 

19K-  1 

91S  -  1 

NR  -  8 

Intel  Analyst 

-  1 

25B  -  1 

NR  -  4 

Pit  SGT 

-  1 

35F  -  1 

Trans  Off 

-  1 

NR 

-  13 

1 .  How  long  have  you  served  in  the  military?  33  months  (mean) 

2.  How  long  have  you  had  an  infantry-related  job?  34  months  (mean)  (N  =  5) 

3.  How  long  have  you  been  a  fire  team  leader?  15  months  (mean)  (N  =  3) 

4.  How  long  have  you  been  a  squad  leader?  10  months  (mean)  (N  =  4) 

5.  How  long  have  you  been  deployed  overseas?  25  months  (mean)  (N  =  10) 

6.  How  long  have  you  been  deployed  in  a  combat  area?  12  months  (mean)  (N  =  7) 

7.  With  which  hand  do  you  most  often  write?  25  Right  _5_  Left 

8.  With  which  hand  do  you  most  often  fire  a  weapon?  26  Right  _4_  Left 

9.  a.  Do  you  wear  prescription  lenses?  15  No  15  Yes 

b.  If  so,  which  do  you  most  often  wear?  9  Glasses  4  Contacts  2  Both 

c.  Which  is  your  dominant  eye?  21  Right  _8_  Left  J_  NR 
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10.  Please  rate  your  skill  level  for  each  of  the  following  activities? 


None  Beginner  Intermediate  Expert 

12  3  4 


ACTIVITY 

MEAN  RESPONSE 

Operating  ground  unmanned  vehicles 

1.54 

Operating  aerial  vehicles 

1.25 

Target  detection  and  identification 

1.79 

Playing  commercial  video  games 

2.79 

Training  with  Army  video  simulations 

2.00 
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Intentionally  Left  Blank. 
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Appendix  B.  Training 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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SAMPLE  SIZE  =  30 


1 .  Using  the  scale  below,  please  rate  the  following  training  features  on  the  Xbox 
Controller: 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


MEAN  RESPONSE 

Length  of  training 

5.70 

Level  of  detail 

5.87 

Hands-on  practice 

6.13 

Overall  quality  of  training 

6.13 

Comments  No.  of  Responses 

Worked  great.  2 

Very  easy  to  use.  4 

User  friendly.  1 

Short,  sweet,  and  to  the  point.  No  extra  fluff.  1 

Easy  to  pick  up.  3 

Liked  using  both  hands.  1 

Good  example.  Allowed  for  as  much  time  as  needed  to  practice.  2 

Patient  instructors.  1 

Easy  to  use  joystick.  1 

Coming  from  a  video  game  background,  the  training  was  very  concise  and  3 
almost  self-explanatory. 

Requires  more  turning  capability.  1 

Doesn’t  have  to  be  as  long.  1 


2.  Using  the  scale  below,  please  rate  the  training  adequacy  that  you  received  in  the 
following  areas  with  the  XBox  Controller: 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


Xbox  Controller 

MEAN  RESPONSE 

Understanding  the  display 

6.10 

Operating  the  controls 

6.43 

Driving  the  robot 

6.17 

Adequate  time  on  the  practice  lane 

6.31 

Clarity  of  instructions 

6.47 

Understanding  of  tasks 

6.50 

Overall  evaluation  of  the  training  course 

6.30 
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Comments  No.  of  Responses 

Great  training.  1 

Easy.  2 

Broke  it  down  to  an  easy  level  of  understanding.  2 

Instructions  were  clear  and  concise.  3 

Well  trained  for  task.  1 


Given  enough  time  and  practice,  any  soldier  could  become  proficient  in  its  1 
use. 


Plenty  of  time  to  practice.  1 

Glare  from  the  sun.  1 


No  more  training  before  testing  would  have  improved  my  initial  experience.  1 
I  felt  I  didn’t  need  too  much  training.  1 

Hard  to  hold.  1 


3.  What  were  the  easiest  and  hardest  training  tasks  to  learn  with  Xbox  Controller? 
Comments  No.  of  Responses 


Easiest 

Everything.  1 

Pick  up  and  go.  1 

I  have  always  had  video  games;  this  was  just  like  it.  1 

Working  joystick.  3 

Speed  control.  4 

Using  controller.  5 

Learning  to  control  the  robot.  1 

Controlling  robot  direction.  3 

Steering.  1 

How  to  move.  1 

The  dead  man’s  switch  function.  1 

Learning  which  button  does  what.  1 

Manipulation  of  the  controls.  1 

Overall  controls  were  easy  to  use.  1 

Maneuvering  robot.  3 

Maneuvering  around  corners.  1 

Driving  forward.  1 

Turning.  1 

Going  in  straight  directions.  1 

Maintaining  speed.  1 

Maintaining  at  low  speed.  1 

Most  soldiers  already  know  how  to  use  the  controller;  not  much  training  is  1 

needed. 

Hardest 

Making  the  movements.  It  takes  getting  used  to.  1 

Not  rushing.  1 

Dealing  with  cords.  1 
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Comments  No.  of  Responses 

Glare  from  sun  (outside  was  difficult  but  still  easier  than  Android).  1 

Judging  how  far  obstacles  were  from  the  robot.  1 

Identifying  course.  1 

Learning  to  control  the  joystick.  2 

Pressing  buttons  correctly.  1 

Holding  both  the  screen  and  the  controller  (while  maintaining  a  weapon).  2 

Hard  to  know  how  to  hold  all  the  components.  1 

Adjusting  to  the  video.  1 

Having  to  use  both  hands.  1 

Turning  on  point.  1 

Turning  response.  1 

Turning  while  stopped.  1 

Turn  while  moving.  1 

A  lot  more  equipment  to  carry.  1 

Maintain  control  at  high  speed.  1 


4.  Using  the  scale  below,  please  rate  the  following  training  features  on  the  Android 
Controller. 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


MEAN  RESPONSE 

Length  of  training 

5.77 

Level  of  detail 

5.83 

Hands-on  practice 

5.83 

Overall  quality  of  training 

5.90 

Comments  No.  of  Responses 

Easy  instruction.  2 

Everything  thing  was  shown  in  a  detailed  manner.  1 

Training  was  professional  and  insightful.  1 

Training  was  good  and  needed.  1 

Instructors  gave  good  input  about  what  worked  for  them.  1 

Light  and  easy  to  carry.  2 

Adequate.  1 

A  challenging  subject  to  train.  1 

Having  previously  had  a  touch  screen,  the  training  made  sense  and  was  1 
clear  and  concise. 

Needed  a  little  more  time  to  familiarize  myself  with  the  controls.  1 


Need  a  lot  more  training  to  achieve  same  goal.  Larger  learning  curve  than  1 
Xbox. 

Tough  to  use;  however,  with  more  fine  turning,  the  Android  may  be  the  one  1 
to  use. 
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Comments  No.  of  Responses 

Hated  it.  Training  was  fine.  1 

Weak  response;  too  sensitive  on  controls  for  operation.  1 

Learning  to  control.  1 

Needs  a  bigger  control  panel.  1 

Using  the  phone  was  made  difficult  by  having  big  thumbs.  1 


5.  Using  the  scale  below,  please  rate  the  training  adequacy  that  you  received  in  the 
following  areas  with  the  Android  Controller: 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


Android  Controller 

MEAN  RESPONSE 

Understanding  the  display 

5.93 

Operating  the  controls 

4.93 

Driving  the  robot 

4.83 

Adequate  time  on  the  practice  lane 

5.80 

Clarity  of  instructions 

6.03 

Understanding  of  tasks 

6.20 

Overall  evaluation  of  the  training  course 

5.87 

Comments  No.  of  Responses 


Excellent  job.  2 

Great  communication  and  explanation  of  controller.  1 

Training  was  good.  2 

Clear  and  concise.  1 

Adequate  training.  4 

Fine.  1 

Liked  having  practice  before  the  actual  run,  1 

It  was  easy,  the  touch  screen  was  personally  harder  to  master  as  a  1 

controlling  system. 

Outside  the  picture  clarity  was  poor.  1 

Controller  just  needs  some  tweaking.  1 

Need  to  have  sensitivity  set  up  to  the  individual  person.  1 

Too  touchy  on  touchscreen  for  the  Android.  1 

Difficult  to  maneuver  because  the  joystick  was  too  small.  1 

Device  was  hard  to  utilize.  1 

I  think  it  might  be  better  to  replace  the  Android  with  an  iPod.  1 

Just  need  a  lot  more  practice.  1 


6.  What  were  the  easiest  and  hardest  training  tasks  to  learn  with  Android  Controller? 
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Comments  No.  of  Responses 

Easiest 

The  way  to  operate  it.  4 

Its  portability.  1 

User  interface.  1 

Understanding  the  controls.  3 

Controlling  the  robot  was  easy.  2 

General  movement  of  the  robot  was  easy.  3 

Driving  at  slow  speeds.  1 

Turning  at  moderate  speed.  1 

Reading  monitor.  2 

The  display  was  simple  and  easy  to  understand.  4 

Backing  up.  1 

Joystick  was  easy  but  it  was  too  small  for  my  fingers.  It  might  work  better  if  it  1 
was  bigger. 

Moving  the  flippers.  1 

One  hand  was  easy  to  use  and  maneuver.  1 

Controlling  adjustable  tracks.  1 

Hardest 

Too  touchy.  Sensitivity  needs  to  be  toned  down.  8 

Controlling  the  touch  control.  A  slight  touch  and  off  course  or  too  fast.  1 

Making  small,  subtle  course  corrections.  It  has  a  tendency  to  over  correct.  1 

Operating  the  Android.  4 

Steering  and  control  of  the  robot.  2 

Keeping  the  robot  on  a  straight  path  at  medium-high  speeds.  1 

Going  forward  and  maintaining  control  at  high  speeds.  1 

Direction  and  speed  control.  3 

Can’t  turn  and  maintain  speed.  1 

Turning  at  slow/fast  speeds.  3 

Operating  the  robot  with  one  finger  on  a  touch  pad.  1 

Small  touch  screen.  1 

Needs  a  bigger  control  panel  area.  1 

Not  hitting  the  other  buttons  on  the  screen  while  trying  to  control  the  robot.  1 
What  makes  the  Android  harder  to  control  is  the  narrowness  of  the  pad.  1 

Coordinating  finger  movement  on  the  touch  screen  with  robotic  movement.  I  1 
was  very  jumpy  for  both  runs. 

Knowing  how  much  time/pressure  to  apply  to  screen.  Gentle  taps  versus  1 

holding  your  thumb  down. 

Making  the  screen  display  work  with  thumb  touch.  1 

The  touch  screen  joystick.  1 

Maneuvering  in  close  spaces.  1 

Difficult  to  back  robot  up  and  return  to  forward.  2 

Adapting  to  new  technology  is  a  challenge.  1 
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Appendix  C.  Post  Iteration 
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SAMPLE  SIZE  =  30 


1 .  Using  the  scale  below,  please  rate  your  ability  to  perform  each  of  the  following  tasks 
based  on  your  experience  with  the  display  that  you  just  used: 

1  2  3  4  5  6  7 

Extremely  difficult  Very  difficult  Difficult  Neutral  Easy  Very  easy  Extremely  easy 


NAVIGATION  TASKS 

MEAN  R 

ESONSE 

Xbox 

Android 

Move  in  the  correct  direction  (outside  unguarded  configuration) 

5.50 

3.27 

Move  in  the  correct  direction  (inside  guarded  configuration) 

5.57 

3.37 

Avoid  obstacles  (outside  unguarded  configuration) 

5.13 

3.30 

Avoid  obstacles  (inside  guarded  configuration) 

5.20 

3.63 

Identify  any  terrain  features  that  might  have  an  adverse  effect 
on  the  ability  of  the  robot  to  maneuver  through  the  course 

4.90 

4.07 

Anticipate  whether  the  turn  radius  of  the  vehicle  will  allow  a  turn 

4.70 

3.70 

Identify  if  you  are  on  the  course 

4.83 

4.32 

Maintain  control  when  driving  at  slowest  speeds 

6.07 

4.30 

Maintain  control  when  driving  at  medium  speeds 

5.53 

3.55 

Maintain  control  when  driving  at  fastest  speeds 

4.47 

2.33 

Return  to  the  correct  route  after  navigating  around  obstacles 

5.07 

3.67 

Overall  ability  to  perform  driving  tasks 

5.50 

3.47 

Comments 


Xbox 


No.  of  Responses 


Great.  1 

Great  training  experience.  1 

Very  easy  to  use.  2 

Much  easier  than  the  Android.  3 

Very  simple.  1 

Easier  mainly  because  I  play  video  games.  1 

Was  familiar  and  toggles  were  easy  to  use  with  practice.  1 

Extremely  easy  to  maneuver  the  robot  on  the  course.  1 

Robot  responded  well.  1 

I  preferred  this  controller.  1 

With  much  less  time  to  practice,  this  form  of  command  can  be  easily  1 

mastered. 

Once  the  driver  adjusts  to  visual  and  speed  controls  and  how  they  relate,  1 
the  robot  becomes  more  maneuverable. 

The  pivot  steer  could  use  a  little  m  ore  power  to  increase  the  speed  of  the  1 
course. 

Speed  control  would  be  helpful.  1 

Inside  track  a  little  slower  response  time  when  turning.  1 

Turning  was  only  problem.  2 

Display  is  too  small,  but  still  manageable.  1 
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Comments  No.  of  Responses 

An  easier  control  to  operate.  1 

Video  console  should  be  mounted  on  controller.  1 

Sensor  on  the  inside  got  in  the  way  at  times  from  making  turns  that  would  1 
have  been  easily  accomplished  manually. 

Easier  inside  for  lack  of  glare  from  sun.  1 

Sometimes  difficult  to  tell  if  you  are  on  course  in  direct  sunlight.  1 

Outside  was  harder  to  identify  course  outlines  in  long  grass.  1 

Difficult  to  see  screen  outside  when  cloudy  (camera  was  not  letting  in  1 

enough  light). 

Difficulty  in  viewing  screen  outdoors  creates  problems  in  keeping  robot  on  2 
course. 

Android 

Enjoyed  the  sensor  for  walls/objects.  1 

Task  performance  was  fairly  easy.  More  practice  will  render  it  easier.  1 

Very  convenient,  i.e.,  it’s  small  and  functional.  With  the  turning,  the  1 

Android  can  be  successful. 

Virtual  joystick  was  difficult  to  adapt  to.  Sometimes  minor  movements  1 

would  achieve  results  and  other  times  extreme  movement  would  be 
required. 

Somewhat  easier  outside  once  I  got  the  hang  of  it.  May  have  been  1 

because  it  was  one  solid  unit  vs.  the  two  units  with  the  Xbox. 

Harder  to  control.  1 

This  controller  seems  incredibly  impractical.  1 

It  was  difficult  to  maintain  a  steady  pace  without  overcorrecting.  1 

Turning  control  unsteady.  1 

Very  difficult  to  turn,  especially  to  the  right.  1 

Very  difficult  to  make  subtle  course  corrections.  1 

Difficult  to  turn  in  own  radius.  1 

Control  was  very  frustrating  to  get  robot  to  turn  properly  as  well  as  to  1 

control  speed. 

Seems  rather  difficult  to  get  the  screen  controls  to  register  actual  position  1 
of  the  user’s  finger. 

Felt  that  I  did  what  was  required  in  a  timely  manner  for  the  amount  of  1 

minimal  training. 

Would  do  well  with  more  practice.  1 

Lack  of  practice  coordinating  finger  movement  on  the  touch  pad,  as  well  as  1 
traveling  at  fast  speeds. 

Turn  down  sensitivity.  2 

Larger  touch  screen.  1 

Hard  to  see  on  screen.  1 

Glare  on  the  phone  screen  outside  made  it  difficult  to  see.  1 

Track  area  on  phone  very  touchy;  did  better  when  I  maintained  contact  with  1 
screen  at  all  times. 

Slight  lag  in  controller  to  vehicle.  Found  controller  challenging  for  me  1 

(never  regularly  use  touch  pad). 
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Comments 

Much  more  difficult  than  the  Xbox. 


No.  of  Responses 

1 


2.  Did  you  experience  any  of  the  following  conditions? 


Number  of  Responses 

Xbox* 

Android 

No 

Yes 

No 

Yes 

Eyestrain 

23 

6 

22 

8 

Tunnel  vision 

28 

1 

27 

3 

Headaches 

29 

0 

29 

1 

Motion  sickness 

29 

0 

29 

1 

Nausea 

29 

0 

29 

1 

Disorientation 

29 

0 

28 

2 

Dizziness 

29 

0 

29 

1 

*One  soldier  did  not  give  a  response  for  the  Xbox. 


Comments  No.  of  Responses 

Xbox 

No  discomfort,  just  unfamiliarity  with  the  system.  1 

Eyestrain  from  glare  from  sun.  4 

Because  of  sun,  it  was  difficult  to  see  the  screen  sometimes  or  even  what  1 
the  camera  was  projecting;  especially  when  it  was  coming  out  of  tunnel. 

When  facing  the  sun  on  emerging  from  the  tunnel,  the  video  feed  seemed  1 
to  have  trouble  adjusting  to  the  light  and  was  frequently  blown  out. 

Eyestrain  from  small  screen.  1 

The  screen  clarity  was  an  issue,  but  I’m  sure  it  could  be  easily  improved.  1 

The  goggle  phone  was  easier  to  see,  the  HTC  phone  outside  was  not  as  1 

clear. 

The  backpack  was  annoying.  1 

Bulky;  tough  to  run  with  weapon.  1 

After  covered  obstacles  camera  took  a  few  seconds  to  adapt  to  light.  1 

Android 

Eyestrain  because  difficult  viewing  screen.  1 

Eyestrain,  though  not  much  true  discomfort,  it  was  just  slightly  difficult  to  1 
orient  the  camera  with  the  robot  when  traveling  at  fast  speeds  due  to  the 
size  of  the  screen. 

Still  hard  to  see  when  going  from  dark  to  light  places.  1 

Screen  a  bit  hard  to  see  in  certain  angles  in  the  sun.  1 

Sensitivity  of  controls.  1 


3.  What  were  the  easiest  and  hardest  training  tasks  to  learn  with  this  controller? 
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Comments 


Xbox 


No.  of  Responses 


Easiest 

Everything.  2 

Very  easy  to  learn  and  use.  2 

User  interface.  1 

If  you  play  video  games,  it’s  pick  up  and  go  with  this  controller.  1 

Joystick.  4 

Maintain  control  at  low  speed.  1 

It  was  significantly  more  maneuverable.  1 

The  controller  was  very  easy  to  manipulate.  3 

Very  easy  to  give  command  to  robot.  1 

Simple  use  of  controller;  minimal  manipulation  needed.  2 

Once  I  had  a  clear  straight  path,  it  was  easy  to  pick  up  the  pace  and  1 

control  the  robot. 

Maneuvering  and  controls  was  simple.  1 

Controllers  functions.  1 

Controlling  direction.  1 

Control  switches.  1 

Turning  was  easiest.  1 

Moving  forward  and  backward.  1 

To  make  it  go  straight.  2 

Easy  to  see.  1 

Nothing  was  easy.  1 

Hardest 

Operating  at  higher  speeds.  2 

Flaps.  1 

Not  pressing  the  flipper  buttons  accidentally.  1 

Getting  used  to  looking  at  screen  on  phone  and  then  correlating  on  1 

controller. 

Seeing  on  the  cell  phone  with  the  glare  outside.  1 

Hard  to  see  in  the  shade.  1 

Not  accidentally  touching  the  Android  screen  and  causing  the  system  to  1 

halt,  and  holding  3  objects  at  the  same  time  ->  screen,  weapon,  and 
controller. 

Identifying  distance  from  robot  to  oncoming  obstacle.  2 

It  was  hard  not  to  rush  through.  All  driving  errors  were  because  of  rushing.  1 
It  was  difficult  to  know  what  movements  to  make  when  the  robot  was  facing  1 
a  light  source. 

Would  be  easier  to  use  if  one  joystick  were  acceleration  and  one  for  turning  1 
like  a  lot  of  driving  games. 

Turning  ability.  1 

Hard  to  turn  and  maintain  any  speeds.  1 

Speed  in  relation  to  control.  1 

Turning  right  or  left  on  point.  1 

Turns  around  trashcans.  1 
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Comments  No.  of  Responses 

Backing  up  and  avoiding  obstacles.  1 

Movement  when  making  sharp  turns.  1 

Slow  turns.  1 

Recognizing  I  was  off  course;  hard  to  see  engineer  tape.  1 

To  maneuver  around  terrain  or  just  not  being  able  to  see  the  engineer  tape  1 
via  the  camera  because  of  the  sun.  This  didn’t  happen  often. 

Hard  to  maintain  with  my  gear.  1 

Moving  from  one  location  to  another  while  carrying  all  required  equipment.  1 
Getting  back  on  track  when  I  went  off  the  course.  1 

Carrying  it  around.  1 

Android 

Easiest 

Concept  was  easy  to  understand.  1 

Using  it.  1 

Controls  were  very  easy  to  get  used  to;  they  are  simple.  1 

Understanding  how  to  move  the  robot.  1 

It  was  just  nice  to  have  a  hand-held  controller.  1 

User  interface.  1 

Getting  back  into  program  after  exiting  was  easier.  1 

Lightweight,  portability.  3 

One  touch  movement.  1 

Camera  worked  great.  1 

Correct  yourself  to  the  path.  1 

Stationary  turning.  1 

Moving  robot  quickly.  1 

Very  little  movement  on  controller  to  navigate.  1 

Learning  speed.  1 

Navigating  at  slow  speeds.  3 

Navigation  was  easy  in  all  directions.  2 

Moving  forward  and  bumper  movements.  1 

Easier  to  use  on  the  outside  obstacle  than  the  inside.  1 

Backing  up  was  easy.  1 

Gradually  turn  at  moderate  speeds.  1 

Learning  what  the  controller  does.  2 

Using  sensor  to  my  advantage.  1 

Nothing  was  easy.  1 

Hardest 

Everything.  1 

Utilizing  the  controller.  1 

Sensitivity  of  controls.  5 

Too  sensitive.  3 

The  toggle  was  either  too  sensitive  or  it  wouldn’t  respond  in  time.  1 

Find  the  sensitivity  on  the  controller  as  in  relation  to  how  robot  actually  1 


moves.  Once  comfortable  with  sensitivity  the  controller  is  very  easy  to 
operate. 
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Comments  No.  of  Responses 

Actually  controlling  it  was  very  difficult.  2 

Direction  of  controls.  1 

Slight  movements  made  robot  go  extremely  fast  making  it  difficult  to  1 

maintain  course. 

Getting  familiar  with  the  turns.  1 

Straight  line  speed  was  even  difficult.  1 

Turning  without  moving  forward  or  backward.  1 

Making  left  and  right  turns;  touchy.  3 

Much  more  difficult  to  turn  right  than  left.  1 

Maintaining  the  correct  line.  Losing  control  of  the  robot  was  fairly  easy  1 
when  on  the  outside.  Inside  was  a  little  easier  to  maneuver. 

Performing  turns  with  robot.  2 

Very  difficult  to  turn  in  place.  1 

The  glare.  1 

Going  forward  at  higher  speeds  and  staying  in  control.  2 

Correct  direction  at  high  speeds.  1 

Maintain  a  consistent  line  at  high  speeds.  1 

Understanding  the  correlation  between  vehicle  performance  and  finger  1 

movement  on  the  touch  screen  at  fast  speeds. 

To  use  the  touch  screen  as  a  controller  versus  a  knob.  It  was  hard  for  me  1 
to  control  the  speed  and  motion  without  the  feeling  or  touch  of  my  finger. 

Getting  it  to  do  it.  1 

I  feel  it  would  take  a  lot  of  practice  before  using  it  in  the  workforce.  1 

Identify  object/course.  1 


4.  Using  the  scale  below,  what  is  your  overall  rating  of  the  controller  that  you  used  this 
iteration? 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


MEAN  RESONSE 

Xbox 

Android 

5.93 

3.72 

Comments 


Xbox 


No.  of  Responses 


Very  good  overall.  1 

Good  side  for  hands.  1 

More  control  made  it  easier  to  adjust.  1 

Controlled  with  ease.  1 

Easy  to  use.  5 

Significantly  more  practical.  1 


I  felt  like  I  had  a  constant  move  the  whole  time  on  this  controller,  and  things  1 
went  much  farther.  This  also  allowed  the  obstacle  avoidance  system  to 
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Comments  No.  of  Responses 

drastically  improve  my  time. 

With  training,  it  would  become  very  easy  to  utilize  robot  for  intended  1 

purposes. 

Using  a  controller  people  are  familiar  with  makes  operating  the  robot  a  1 
simple  task. 

Works  great  if  we  can  work  on  the  stationary  turning.  1 

With  exception  of  flipper  buttons,  quite  easy  to  control.  1 

Controller  was  easy  to  move  robot;  however,  it  was  somewhat  2 

cumbersome  to  carry  weapon  and  controller  and  camera  simultaneously. 

Also,  back  pack  may  create  difficulty  with  carrying  combat  load. 

Clunky  when  outside  with  phone  (display)  controller,  and  backpack.  1 

Two  joysticks  would  be  better.  1 

Easier  inside  than  outside.  1 

Perhaps  attach  the  screen  to  the  Xbox.  1 

Small  screen,  but  very  clear.  1 

Traveling  with  cord  was  issue.  Could  detach  from  bag  (but  great  that  it  is  a  1 
quick  reconnect). 

Only  concern  re:  light  connect  that  automatic  makes  it  appear  like  a  1 

“tunnel”  which  causes  poorer  visibility. 

More  practice  needed.  1 

Android 

It  was  fun.  1 

The  compactness,  weight  make  it  easy  to  take  and  move  it  with  you.  2 

Much  more  efficient  to  carry.  1 

Controller  is  very  good.  1 

Some  people  need  more  time  to  adjust  to  it.  1 

If  you  gain  experience  in  using  it,  you  can  master  it.  5 

For  someone  with  limited  experience,  it  is  a  trial.  1 

Think  I  might  do  better  with  a  game  control.  Something  more  hands-on.  1 

Too  difficult  with  finger  touch.  Prefer  a  more  hands-on  system.  1 

Would  be  better  with  practice.  1 

It  wasn’t  bad,  but  the  Xbox  was  easier.  1 

Not  the  best  one  for  my  background,  though.  I  did  adapt  and  get  more  1 
proficient  as  the  minutes  went  on. 

Screen  is  a  bit  small,  but  gets  the  job  done.  1 

Other  than  getting  comfortable  with  the  sensitivity  of  the  Android,  it  was  1 
really  effective. 

Easier  outdoors  than  indoors.  1 

I  like  the  ease  of  just  carrying  a  phone,  but  I  would  make  the  control  1 

surface  a  little  larger  and  less  sensitive. 

I  think  you  should  consider  the  Android  X  instead  of  the  Android  HTC.  X  1 
has  4.3”  touch  screen. 

Very  high  tech,  but  incredibly  impractical.  1 

I  believe  if  sensitivity  is  altered  to  the  operator,  the  Android  controller  could  1 
be  very  effective. 
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Comments  No.  of  Responses 

Very  touchy,  which  leads  to  problems  when  precision  is  needed.  1 

Too  sensitive.  Not  recommended  for  use  in  combat  zone.  1 

I  thought  the  joystick  control  was  a  little  too  sensitive.  1 

Perhaps  being  able  to  view  the  phone  horizontally  instead  of  vertically  1 

and/or  using  a  stylus  might  make  control  easier. 

Too  difficult  to  control  speed  as  well  as  turns.  1 

At  times  hard  to  maneuver  when  I  wanted  it  too.  1 

Difficult  to  make  minor  course  corrections.  1 

Exit  and  menu  buttons  should  be  smaller  when  using  the  steering  button.  1 
The  absence  of  buttons  made  it  a  lot  harder  to  control.  1 

Ran  into  signal  connection  errors  as  well.  1 

Would  rather  use  on  obstacles  where  I  would  have  more  room  to  1 

maneuver. 
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Intentionally  Left  Blank. 
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Appendix  D.  End  of  Experiment 
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SAMPLE  SIZE  =  30 


1 .  Which  controller  did  you  prefer  overall? 


Number  of  Responses 

Xbox  Controller 

Android  Controller 

No  Preference 

26 

2 

2 

Comments  No.  of  Responses 

Xbox 

Xbox  easier  to  use.  7 

Just  works  better.  1 

Easier  to  operate.  1 

There  was  something  physical  to  hold  while  steering.  1 

Easy  to  control/manipulate.  5 

Xbox  make  operating  robot  a  simple  task.  2 

More  maneuverability.  1 

Reacts  instantly  to  your  actions.  1 

More  tangible.  2 

More  natural.  2 

A  lot  smoother.  1 

Did  not  require  multiple  inputs  to  function  properly.  1 

Joystick  was  natural  to  use;  made  the  difference.  3 

Easier  to  control  speed,  direction,  and  movement.  7 

Turning  was  clearer.  1 

Felt  more  accurate  outcome  of  intended  movements.  1 

More  responsive  and  less  room  for  error.  1 

Has  great  sensitivity.  2 

Even  with  the  backpack,  it  was  significantly  easier  than  the  Android.  1 

Am  used  to  this,  like  when  playing  games.  1 

Familiar  with  remote  for  Xbox.  1 

Only  negative  is  the  equipment  and  cords  you  had  to  carry.  1 


No  preference 

Android  worked  better  outdoors,  I  believe,  when  you  kept  constant  1 

pressure  on  the  touch  screen. 

I  would  rather  use  the  Android  on  outside  obstacles  where  I  would  have  1 
more  room  to  maneuver  and  use  the  Xbox  in  more  restricted  areas. 
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2.  Do  you  have  any  suggestions  for  ways  to  increase  the  effectiveness  of  the  Xbox 
Controller? 


Comments  No.  of  Responses 

Really  effective.  1 

For  outside,  find  a  way  to  make  it  all  together  with  screen.  1 

Find  some  way  to  mount  the  phone  onto  the  controller.  1 

Attach  screen  to  the  remote.  3 

Integrate  screen  into  controller  to  keep  from  having  to  carry  two  devices  2 
along  with  the  weapon. 

Get  the  screen  attached  somehow  or  make  computer  in  bag  be  the  monitor  1 
as  well. 

If  it  could  have  a  screen  attached  to  it,  as  opposed  to  using  the  phone  as  1 
the  screen,  it  would  make  maneuvering  a  lot  better. 

Less  cords.  Can  you  do  wireless?  1 

Redesign  with  only  features  used  for  movement  of  robot  and  also  add  a  1 
screen. 

Larger  viewing  screen  (Droid  x  4.3”  screen).  1 

Speed  control.  1 

Eliminate  extraneous  buttons  and  shrink  size  of  controller.  2 

Make  the  buttons/joystick  a  little  larger  could  make  easier.  1 

Use  a  joystick  with  different  buttons  place  on  top  of  it.  1 

Use  both  joysticks,  one  for  steering  and  one  for  acceleration.  1 

Xbox  should  be  autonomous.  1 

Work  on  stationary  turning.  1 

Definitely  use  both  joysticks.  One  for  the  left  track  and  one  for  the  right.  2 

Maybe  would  be  easier  to  operate  with  the  use  of  both  toggles.  Left  1 

toggle-directional,  right,  forward  and  back  (throttle). 

T urning  buttons  left  and  right  for  top  of  controller.  1 

Find  a  way  to  place  flipper  buttons  where  my  fingers  won’t  touch  them  1 

(control  design).  Do  you  need  a  button  (green  one)  to  enable  joystick  to 
work? 

In  a  field  environment  it  might  be  easier  if  the  controller  and  screen  could  1 
be  attached  to  one  another. 

Smaller  pack  for  the  back.  1 

Make  it  a  one-handed  device  so  we  can  ditch  the  backpack.  1 

Losing  the  backpack  would  be  a  plus,  but  not  a  must  to  affect  performance.  1 

3.  Do  you  have  any  suggestions  for  ways  to  increase  the  effectiveness  of  the  Android 
Controller? 

Comments  No.  of  Responses 

Is  good,  it  just  requires  more  practice.  1 

It’s  impractical  and  the  screen  isn’t  uniformly  sensitive.  1 

Include  a  stylus.  1 

Very  frustrating  user  interface.  1 
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Comments  No.  of  Responses 


Needs  to  be  more  responsive.  Too  much  delay  when  turning  and  driving  at  1 
the  same  time. 

Fine  tune  the  toggle  screen  or  have  real  buttons.  1 

Bigger  touch  screen.  5 

Increase  the  area  where  the  finger  has  control  over  the  steering.  With  1 

such  a  small  area,  those  of  us  with  large  fingers  had  a  harder  time 
steering. 

Make  the  controls  bigger.  1 

Make  buttons  (exit,  menu)  smaller  so  they  aren’t  accidentally  activated  1 
when  going  left  and  right. 

Larger  buttons  to  maneuver.  Rather  crowded  on  screen.  Worried  about  1 
hitting  flipper  buttons  when  going  left. 

Less  sensitivity.  7 

Sensitivity  needs  to  be  better  calibrated.  3 

Probably  be  more  effective  if  calibrated  to  operator.  1 

Sensitivity  of  the  controller  needs  to  be  better.  Sometimes  when  using,  I  1 
had  difficulties  controlling  the  robot  effectively. 

Decrease  the  sensitivity  of  the  tracking  of  the  track  pad.  1 

Controls  were  much  too  sensitive  with  too  little  difference  in  finger  position  1 
between  barely  moving  and  full  speed. 

Make  it  not  so  touchy  and  jumpy.  I  did  better  when  I  maintained  contact  1 
with  screen  at  all  times. 

More  range  of  motion  for  control  of  movements.  1 

Regulate  speed.  2 

Adjust  left  and  right  mounts.  1 

Turning  needs  to  be  perfected;  hard  to  control.  2 

Make  the  digital  joystick  larger  because  if  you  moved  your  finger  too  far  1 
over  you  would  lose  controls. 

Don’t  use  them.  2 


4.  Which  controller  did  you  prefer  when  performing  the  following  tasks? 


No.  of  Responses 

Xbox 

Android 

Navigating  around  obstacle 

30 

0 

Driving  straight  along  course  (s) 

27 

3 

Driving  through  a  turn 

28 

2 

Manning  the  robot  using  bounding  technique 

19 

11 

Maneuvering  the  robot  through  the  building 

29 

1 

Negotiating  the  robot  through  confined  areas 

30 

0 
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5.  When  considering  the  advantages  and  disadvantages  of  a  one-handed  vs.  two- 
handed  operation,  which  do  you  prefer  and  why? 


Number  of  Responses 

One-handed 

Two-handed 

Both 

NR 

9 

19 

1 

1 

Comments  No.  of  Responses 

One-handed 

Is  better  in  combat  or  a  high  stress  situation  because  you  have  a  free  hand  1 
to  do  other  things. 

It  frees  my  other  hand  to  use  the  radio,  if  necessary,  or  to  carry  my  3 

weapon,  etc. 

Android  is  easier  to  use  one-handed  since  the  screen  is  on  the  controller.  1 
Allows  me  to  access  my  weapon  more  easily.  1 

Can  keep  hidden  better.  1 

More  convenient.  1 

Better  when  fired  upon.  1 

Would  allow  for  quicker  recovery  of  firearm  or  to  open  doors,  etc.  1 

Easier  mobility,  especially  with  a  full  combat  load.  1 

Cumbersome  to  carry  weapon  pack  and  controller.  1 

With  practice  one  hand  is  better.  1 

Hard  to  control  multi-directional.  1 

Second  hand  can  be  on  my  rifle  for  easier  transition  in  battle.  1 

Rifle  must  be  readily  available  in  combat  zone.  1 

Two-handed 

Familiarity  with  the  Xbox  makes  it  easier.  1 

Xbox  is  easiest  to  use  by  far.  1 

Allowed  more  control.  9 

Allows  for  more  options.  1 

I  feel  people  would  be  more  confident  when  using  the  Xbox  controller.  1 

Easier  with  movements.  2 

Allows  more  positive  control  over  steering  method,  keeps  a  steady  screen  1 
and,  therefore,  better  steering. 

More  control.  If  I  am  maneuvering  something  in  combat  like  this,  I  will  still  1 
be  consumed  whether  it  be  one  or  two-handed. 

Gave  you  something  to  hold  on  to  for  more  stability.  2 

Felt  more  confident.  1 

More  support  means  more  control  and  more  fine  touch.  1 

Stability  to  operate  control.  If  controller  was  attached  to  equipment,  then  1 
might  be  able  to  do  one-handed. 

You  maintain  a  better  control  over  body  motion  and  controller.  Breathing  1 
and  more  fluid  movements  with  controller  are  better  maintained. 

You  have  the  ability  to  trigger  multiple  options  at  the  same  time  as  well  as  1 
a  more  stable  controller/screen. 

As  long  as  you  have  a  buddy  holding  a  weapon  is  not  a  problem,  and  with  1 
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Comments 


No.  of  Responses 


two  hands  you  are  better  able  to  control/space  controls  than  a  small 
compact  one-handed  system. 

Pressure  depends  on  two  hands. 

Helps  with  changing  direction. 

Helps  do  different  tasks  to  increase  mobility. 

You  don’t  have  hand  free  for  weapon  or  if  you  need  it. 

Android  is  autonomous. 


6.  Using  the  scale  below,  rate  your  thoughts  on  the  importance  of  having  one  hand  free. 


1 

Don’t  need 
at  all 


2 

Could  do 
without 


Not  important 


4 

Neutral 


5 

Important 


6 

Very 

important 


Extremely 

important 


MEAN  RESPONSE 

4.47 


Comments  No.  of  Responses 

You  have  a  weapon  and  need  to  be  able  to  shoot  it.  1 

Two  hands  mean  someone  would  be  dedicated  to  the  robot.  Security  1 

could  be  an  issue. 

Great  test.  Just  need  to  find  a  way  to  have  Xbox  controller  complete  with  1 
display. 

Whether  using  one  or  two  hands,  controller  would  still  have  to  be  put  down  1 
or  away  before  using  weapon.  Ideally,  weapon  use  for  the  person 
controlling  the  robot  would  be  minimal. 

One  hand  free  is  not  important.  I  will  still  need  to  drop  the  controller  in  one-  1 
handed  situations  to  return  fire. 

If  you  incorporate  this  into  your  SOPs  and  or  combat  drills,  you  will  learn  1 
how  to  manipulate  this  system  while  using  your  primary  and  secondary 
weapon. 

In  the  field,  it  is  always  important  to  be  able  to  fire  your  weapon,  move  and  1 
be  aware  of  the  situation  at  all  times.  When  moving  the  robot,  you  tend 
to  focus  on  the  act  alone  and  things  around  you  become  less 
acknowledgeable.  Have  one  hand  free  would  be  great,  but  2  hands 
without  cords  would  help  maintain  more  control. 

I  would  have  to  test  in  more  realistic  battlefield  environments  before  I  could  1 
give  an  educated  answer  as  to  whether  it  would  be  important  to  have  a 
hand  free  during  testing.  At  this  point,  I  feel  that  the  increased  control 
over  steering  with  two  hands  outweighs  the  importance  of  having  one 
hand  free. 

It  would  probably  be  more  convenient  but  I  felt  better  control  with  two  1 

hands. 

Takes  a  lot  of  concentration  to  use  vehicle  with  all  the  devices.  One  hand  1 
free  is  not  going  to  help  you  fire  upon  the  enemy  while  in  use.  Maximize 
the  efficiency  and  effectiveness  of  the  device,  whether  it  requires  one 
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Comments  No.  of  Responses 

hand  or  two. 

You  are  not  likely  to  be  fighting  and  controlling  the  Android  at  the  same  1 
time.  You  will  probably  be  behind  cover  controlling  it,  meaning  you  can 
use  two  hands. 

Though  having  a  free  hand  would  be  good,  your  eyes  would  still  be  on  the  1 
screen. 
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Intentionally  left  blank. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


ARL 

CISD 

HRED 

LADAR 

NASA-TLX 

ocs 

ocu 

PDA 

TOC 


U.S.  Army  Research  Laboratory 
Computational  and  Information  Sciences  Directorate 
Human  Research  and  Engineering  Directorate 
laser  detection  and  ranging 

National  Aeronautics  and  Space  Administration-Task  Load  Index 

Officer  Candidate  School 

operator  control  unit 

personal  digital  assistant 

tactical  operations  center 
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NO.  OF 

COPIES  ORGANIZATION 


1  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
only)  DTIC  OCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BELVOIR  VA  22060-6218 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  HRR 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
RDRL  CIO  LL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
RDRL  CIO  MT 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
RDRL  D 

2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 
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NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HRM  A  J  MARTIN 
MYER  CENTER  BLDG  2700  RM  2D3 1 1 
FORT  MONMOUTH  NJ  07703-5601 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  C  A  DAVISON 

320  MANSCEN  LOOP  STE  115 
FORT  LEONARD  WOOD  MO  65473 

2  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  DI 

T  DAVIS 

J  HANSBERGER 

BLDG  5400  RM  C242 

REDSTONE  ARSENAL  AL  35898-7290 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HRS  EA  DR  V  J  RICE 
BLDG  4011  RM  217 
1750  GREELEY  RD 
FORT  SAM  HOUSTON  TX  78234-5002 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HRM  DG  K  GUNN 
BLDG  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ARMC  FIELD  ELEMENT 
RDRL  HRM  CH  C  BURNS 
THIRD  AVE  BLDG  1467B  RM  336 
FORT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 

AWC  FIELD  ELEMENT 
RDRL  HRM  DJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FORT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HRM  CK  J  REINHART 
10125  KINGMAN  RD  BLDG  317 
FORT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HRM  AY  M  BARNES 
2520  HEALY  AVE 
STE  1172  BLDG  51005 
FORT  HUACHUCA  AZ  85613-7069 


1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HR  MP  D  UNGV ARSKY 
POPE  HALL  BLDG  470 
BCBL  806  HARRISON  DR 
FORT  LEAVENWORTH  KS  66027-2302 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  DQ  MR  FLETCHER 
NATICK  SOLDIER  CTR 
AMSRDNSCWSE  BLDG  3  RM  343 
NATICK  MA  01760-5020 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AT  J  CHEN 
12350  RESEARCH  PKWY 
ORLANDO  FL  32826-3276 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AT  C  KORTENHAUS 
12350  RESEARCH  PKWY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AS  C  MANASCO 
SIGNAL  TOWERS 
BLDG  29808A  RM  303A 
FORT  GORDON  GA  30905-5233 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  CU 
6501  E  11  MILERD  MS  284 
BLDG200A  2ND  FL  RM2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
FIRES  CTR  OF  EXCELLENCE 
FIELD  ELEMENT 
RDRL  HRM  AF  C  HERNANDEZ 
3040  NW  AUSTIN  RD  RM  221 
FORT  SILL  OK  73503-9043 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AV  S  MIDDLEBROOKS 
91012  STATION  AVE  RM  348 
FORT  HOOD  TX  76544-5073 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  CN  R  SPENCER 
DCSFDI  HF 

HQUSASOC  BLDG  E2929 
FORT  BRAGG  NC  283 10-5000 
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NO.  OF 

COPIES  ORGANIZATION 


1  ARMY  RSCH  LABORATORY  -  HRED 
HUMAN  RSRCH  AND  ENGRNG 
DIRCTRT  MCOE  FIELD  ELEMENT 
RDRLHRMDW  E  REDDEN 
6450  WAY  ST 
BLDG  2839  RM  310 
FORT  BENNING  GA  31905-5400 

1  ARMY  G1 
(CD  DAPE  MR  B  KNAPP 
only)  300  ARMY  PENTAGON  RM  2C489 
WASHINGTON  DC  20310-0300 


ABERDEEN  PROVING  GROUND 

4  DIR  USARL 
RDRL  HR 
L  ALLENDER 
T  LETOWSKI 
RDRL  HRM 

P  S AV AGE-KNEP  SHIELD 
RDRL  HRS  D 
B  AMREIN 
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